

Quality Management Workflow

Annotator Performance Management

Analyze performance of every annotator for every image or video-frame. Following statistics are available:

1. Number of labels created by an annotator.
2. Time spent per label (average) in seconds.
3. Total time spent on the project in seconds.
4. Last activity by every annotator.
5. Reviewers can accept or reject annotations of every annotator. Reviewer acceptance rate is a percentage that is calculated by aggregating reviewer's acceptance count.
6. Reviewers can optionally give a score for every image. Reviewer's average score (between 0 and 10) is calculated
7. Consensus using IOU

Review Mode: How to review a task performed by Annotators

In Review mode performance of an annotator can be reviewed for every image or video-frame. As seen in video below reviewers can

1. "Accept" "Reject" an annotation: Mandatory
2. Give a score between 0 and 10, (optional).
3. Leave a comment for the annotator (optional).

Annotation Mode: How to create annotations

From project settings page select button "Start Annotating".

1. Select classes on right panel
2. When a class is selected, corresponding tool is activated.
3. Draw annotation with respective tool
4. Enter questionnaire related to the annotations
5. Press Button:"Save" to save annotation
6. Press Button:"Delete" to delete an annotation.
7. Press left button or right button to move left and right.